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Locally enhancing display information 



The invention relates to a system for enhancing a part of the display 
information corresponding to an area of a display screen of a display apparatus, to a method 

of locally enhancing display information, to a computer with provisions for locally enhancing 
display information, and to a display apparatus with provisions for locally enhancing display 
information. 

WO-A-99/20042 discloses an image display device m which the drive signal 
range of a cathode ray tube is locally enhanced in a window. The average luminosity in the 
window is monitored and the drive signal range is reduced if there is a danger of doming in 
the wmdow. The windows in which the amplification gain is increased can be selected. In 
one version, this is done by the image source which is, for example, a WEB browser which 
shows a WEB page containing text and subwindows with photos or video. The WEB browser 
may indicate one or more of the subwindows with photos or video in order to increase the 
amplification gain. Alternatively, this is possible by determining the high spatial firequency 
energy-content in a window, which would be lost by increasing the gain due to increased spot 
size m the CRT, and comparmg it with the low spatial firequency energy-content. 

It is an object of the invention to provide a system of a display information- 
generating device and a display apparatus for displaying the display information, which 
system has a more reliable way of automatically detecting which window should be enhanced 
for a larger variety of situations. 

To this end, a first aspect of the invention provides a system for enhancing a 
part of the display information corresponding to an area of a display screen of a display 
apparatus, as defined in claim 1 . A second aspect of the invention provides a method of 
locally enhancing display information as defined in claim 1 1. A third aspect of the invention 
provides a computer wilh provisions for locally enhancing display information as defined in 
claim 12. A fourth aspect of the invention provides a display apparatus with provisions for 
locally enhancing display information as defined in claim 13. Advantageous embodiments of 
the invention are defined in the dependent claims. 
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The system comprises a display information-generating device, for example a 
computer, and a display apparatus, for example a monitor, for displaying the display 
information generated by the display information-generating device on a display screen. The 
system further comprises a detector for detecting whether a part of the display information 
corresponding to an area on the display screen has to be enhanced. The part of the display 
information may be, for example, a Windows® window. The word "Windows" starting with 
a capital letter refers to the Microsoft operating system Windows®. The word "windows" 
starting with a small letter refers to an area of display information as displayed on a screen of 
a display device. This area may have a rectangular or any other shape. 

The detector detects whether in the part of the display information: 

(i) an application is one of a group of applications indicating that non-synthetic 
information is displayed, in which the application is not a picture viewer, or 

(ii) an extension of a file is one of a group of extensions indicating that non- 
synthetic information is displayed, or 

(ill) moving information is displayed. 

If at least one of these criteria is true, the display information in the area will be enhanced. 

Synthetic information is computer-synthesized information such as, for 
example, text, graphics, and tables, and non-synthetic information is not computer- 
synthesized information, for example, photographs and video/film. 

With respect to (i), it may be detected whether the application which provides 
the information in the area is an application for showing photographs or video. The non pre- 
published application WOOl/411 17 (attorneys docket PHN17.784), discloses a system for 
improving the image quality of selected video windows when the video information in the 
window is a photograph or movmg video. More specifically, it is disclosed which 
information has to be transported from the computer to the monitor to enable the monitor to 
determine the exact position of the window on the display screen. It is further disclosed that 
the microprocessor in the computer may detect which application is running in a window, 
and if this application is a picture viewer, indicate to the monitor that a picture is displayed. 
Therefore, a picture viewer has to be disclaimed. Examples of applications which fall within 
the group of applications indicating that non-synthetic information is displayed, and which 
are not picture viewers, are movie players or other applications for showing video, for 
example from a TV tuner card or from a video recorder, camcorder, or digital (video) camera 
connected to the PC. 
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With respect to (ii), it may be detected whether an extension of a file which 
contains the information to be displayed in the area is one of a group of extensions indicating 
that non-synthetic information is displayed. Examples of such extensions are: jpg, tiff, mpg, 
mov and so on. 

5 With respect to (iii), it is detected whether moving information is displayed. 

Al kinds of known algorithms or circuits may be used to determine whether the information 
displayed in the area comprises moving objects, such as are present in movies. 

Some examples of enhancing of the display information are: an increased light 
output ( a higher brightness or contrast), an increased sharpness, an adapted white color, a 

1 0 noise reduction, or any other signal processing improving the quality of the display 
information in the area, or drawing the attention to the area. 

In an embodiment of the invention as defined in claim 4, the movement in the 
area is determined by comparing the video data in the area stored at an earlier instant with the 
corresponding video data at a later instant. If the differences between both video data is larger 

1 5 than a predetermined limit value, it is very likely that movement has occurred. It is, for 
example, possible to compare the values of each data word of the stored data with the 
corresponding data word of the later data. If more than a predetermined number of 
corresponding data words differ more than a predetermined value, it is likely that motion is 
present. An efficient embodiment of a motion detector is defined in claim 5. Now, the 

20 absolute values of the differences of the corresponding data words are summed. If the value 
of this sum exceeds the limit value, motion is most probably present. 

The motion detection can be improved by continuously keeping track of the 
difference between corresponding data words. Motion is only detected if this difference is 
almost continuously larger than the limit value. It is not required to compare all the data 

25 words in the area, a subset might be sufficient. For example, such a subset might comprise a 
few lines of data words only, or scattered groups of pixels, or every third pixel. 

In an embodiment of the invention as defined in claim 8, the information- 
generating device provides coordinates defining the area to the display apparatus. The display 
apparatus comprises the detection circuit. The detection circuit comprises an integrator, a 

30 sample-and-hold circuit, and a comparator. At a first instant, the detection circuit determines 
an intensity value of a line or a sum of lines in the area. The determined intensity value at the 
first instant is stored by the sample-and-hold circuit. The comparator compares the stored 
intensity value with a fiirther intensity value of a corresponding line or a sum of lines in the 
area at a later instant and indicates whether a difference between the stored intensity value 
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and the further intensity value exceeds a limit value. In this way, the detection whether the 
area contains moving video is performed in the monitor. The monitor receives the 
coordinates of all areas (windows) to be displayed and decides which areas will be enhanced 
and which will not be enhanced. The computer does not have to detect which areas have to be 
enhanced and to send only the coordinates of areas to be enhanced to the monitor, the 
computer sends the coordinates of all windows to the monitor. This is especially relevant m 
situations wherein the load of the computer processor should be minimized. 

In an embodiment of the invention as defined in claim 9, the video processing 
automatically enhances the video information in the area when one or more of the criteria (i), 
(ii), or (iii) is applicable to the video information displayed in the area. 

In an embodiment of the invention as defined in claim 10, a user may indicate 
that an area which is not yet enhanced should be enhanced, or that an automatically enhanced 
area should not be enhanced. In the latter situation, the user overrules the automatic 
enhancement fimction. 

These and other aspects of the invention become apparent firom and will be 
elucidated with reference to the embodiments described hereinafter. 

In the drawings: 

Fig. 1 shows examples of areas on the display screen which correspond to 
parts of the information. 

Figs. 2 to 5 show a block diagram of an embodiment of the system in 
accordance with tihie invention. 

Fig. 6 shows a flow chart elucidatmg an embodiment of the method in 
accordance with the invention, and 

Figs. 7 to 9 show an embodiment in accordance with the invention of a 
detection circuit for detecting movement in an area. 

Fig. 1 shows examples of areas 1, 2, 3 on the display screen DS, which areas 
correspond to parts of the display information DI. As an example, only rectangular 
Windows® windows are shown. The windows might be non-overlapping as shown by 
window 1, or overlapping as illustrated by window 2 which overlaps window 3. The part 1, 2, 

3 of the display information DI to be enhanced corresponds to the area within the rectangle 
indicating the areas 1 and 2, or may correspond to a non-rectangular area as indicated by the 
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hatched area of window 3. The invention is of course also usable when non-rectangular 
windows or areas are concerned. 

Figs. 2 to 5 show a block diagram of an embodiment of the system in 
accordance with the invention. 

Fig. 2 shows a system of a computer PC and a monitor MON. The computer 
PC may be a personal computer, the monitor MON may comprise a cathode ray tube, a liquid 
crystal display, or any other display device with a display screen DS which displays the 
display information DI. The PC comprises a graphics adapter GA, which supplies the display 
information DI to a detection circuit DEI, and an Encoder ENC. The encoder ENC, and the 
corresponding decoder DEC in the monitor MON are not essential and can be omitted. A 
processor PRO, for example a microprocessor, controls the graphics adapter GA to supply 
the display information DI, and controls the detection circuit DEI to supply a control signal 
CIl to mdicate whether at least one of the following criteria (i) to (iii) is true in the part 1, 2, 
3 of the display information DI: 

(i) an application is one of a group of applications indicating that non-synthetic 
information is displayed, in which the application is not a picture viewer, or 

(ii) an extension of a file is one of a group of extensions indicating that non- 
synthetic information is displayed, or 

(iii) moving information is displayed. 

The processor PRO provides coordinates CO of the areas to be enhanced or 
highlighted to the detection circuit DEI and to the encoder ENC. The encoder ENC combines 
the display information, the coordinates CO, and the control information CIl to obtain an 
encoded signal EDI which is transported from the computer PC to the monitor MON. 

In this embodiment in accordance with the invention, the computer PC detects 
whether the display information DI in a certain area indicated by the coordinates CO has to 
be enhanced or not. The computer PC sends the coordinates CO of areas to be enhanced to 
the monitor MON. The monitor MON comprises the enhancement circuitry. This system has 
the advantage that the decision whether an area has to be enhanced is taken in the computer 
PC where the information on the position of the area is available easily, and where the 
processing power of the available processor can be used to perform the detection algorithm. 

The monitor MON comprises the decoder DEC which decodes the encoded 
display signal EDI into the display information DI, the control signal CIl, and the 
coordinates CO. An enhancing circuit EMI enhances the areas defmed by the coordinates CO 
when the control signal CIl indicates to do so. The enhanced video information which is 
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enhanced in the areas is supplied as the drive videosignal DSI to the display screen DS via a 
drive circuit DRS. The enhancement or highlighting may be an increased light output, an 
adapted white color, a sharpness improvement, a noise reduction or any other signal- 
processing algorithm which differs for the area to be enhanced with respect to areas which 
5 are not enhanced. 

If the computer PC and the monitor MON are separate xmits interconnected via 
a standard interface and interface cable, such as the VGA interface and cable, the encoder 
ENC may combine the display information DI, the control signal CIl and the coordinates CO 
such that the encoded information EDI can be transported via the standard interface. Many 

10 ways of encoding are known, for example, the control information may be encoded on the 
video signal during the blanking period, or on the synchronization signal(s) during the scan 
period, or during the scan period on a limited number of lines of the video information. It is 
also possible to supply the control signal CIl and the coordinates CO to the monitor MON 
via a separate line or via a USB or other bus. If the computer PC and the monitor MON are 

1 5 present in a single unit, the encoder and decoder may be replaced by any other way of 
providing the information. 

A flow chart explaining the operation of an embodiment of the system in 
accordance with the invention is described with reference to Fig. 6. Embodiments of the 
detection circuit DEI which detect motion in the area are described with reference to Figs. 7 

20 to 9. 

Fig. 3 shows a system of a computer PC and a monitor MON in accordance 
with another embodiment of the invention. In this embodiment, both the detection and the 
enhancement are performed in the computer PC. In this way, no control information and no 
information on the position of the areas has to be supplied to the monitor MON. 
25 The PC comprises a graphics adapter GA, which supplies display information 

DI to the enhancement circuit EM2 and to the detection circuit DE2. The detection circuit 
DE2 supplies a control signal CI2 to the enhancement circuit EM2 to indicate whether at 
least one of the criteria (i) to (iii) is fulfilled in the part (1, 2, 3) of the display information 
(DI). 

30 The display signal VS supplied to the monitor MON already comprises the 

enhanced areas. 

The monitor MON comprises the drive circuit DRC and the display screen DS and does not 
require an enhancement circuit and the signals to control it. 
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Fig. 4 shows a system of a computer PC and a monitor MON in accordance 
with another embodiment of the invention. In this embodiment, both the detection DE3 and 
the enhancement EM3 are performed in the monitor MON. The computer PC supphes the 
coordinates CO of all areas (windows) to the monitor MON. In this way, the extra circuitry in 
5 the computer and/or the load on the processor PRO of the computer PC is minimized. 

The PC comprises a graphics adapter GA which supplies display information 
DI to the optional encoder ENC under the control of the processor PRO. The processor PRO 
further supplies the coordinates CO of all the areas to be displayed to the encoder ENC. The 
encoded display signal EDI is supplied to the monitor MON. The monitor comprises the 
10 decoder DEC and the enhancement circuit EMS. The decoder DEC decodes the display 
information DI and the coordinates CO from the encoded display information EDI. The 
detection circuit DE3 receives the display information DI and the coordinates CO to supply a 
control signal CI3 to the enhancement circuit EMS to indicate whether at least one of the 
criteria (i) to (iii) is fulfilled in the part (1 , 2, 3) of the display information (DI). The 
1 5 enhancement cncuit EMS will enhance the display information in the area if the control 
signal CIS indicates to do so. The display signal vwth enhanced area(s) is displayed on the 
display screen DS via a driver circuit DRC. 

Fig. 5 shows another embodiment of a computer PC. The computer PC fiarther 
comprises an input circuit IM and a control circuit CON. The part of Fig. 5 which has been 
20 described with reference to Fig. 2 is not repeated. 

The input circuit IM receives user input UI supplied by a user with an input 
device (not shown, for example, a mouse or a touch screen) to supply user information UCl 
to the processor PRO. The processor PRO supplies the user command UC2 to the control 
circuit CON to indicate whether the part 1, 2, 3 of the display information DI should be 
25 enhanced or not. It is also possible that the input circuit IM supplies the user information 
UCl as a command to the control circuit CON directly. 

The control circuit CON receives the control signal CIl from the detection 
circuit DEI and the user information UC2 to supply an adapted control signal CIl' to activate 
or deactivate the enhancing in correspondence with the user input, independent of the 
SO automatic detection by the detection means DEI . In this way, the user is able to overrule the 
automatic detection by indicating which window which was not detected to be enhanced has 
to be enhanced, or which window which was detected to be enhanced should not be 
enhanced. 
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Fig. 6 shows a flow chart elucidating an embodiment of the method in 
accordance with the invention. Software (further referred to as HighLight or HL-application) 
in the computer PC running on the processor PRO selects all possible candidates of areas 
(further referred to as windows) to be highlighted. In step 1 , the HL-application detects 
whether a window is opened, for example by regularly querymg all existing wmdow IDs as 
available in Windows (Windows with a capital letter refers to the operatmg system Microsoft 
Windows®). In step 2, it is detected (for example, from the window ID) which application is 
associated with the window (for example, mediaplayer.exe), and whether this application is 
one of the group of appUcations for which it is very likely that non-synthetic information is 
displayed in the window. In step 3, it is detected (for example, from the window ID, because 
the window title of the appUcation displays the opened file, for example my movicmpeg) 
which file extension the file opened in the window has (for example, mpeg), and whether this 
extension is one of a group of extensions for which it is very likely that non-synthetic 
information is displayed in the window. In step 4, it is detected whether the information 
displayed in a window is moving. In the embodiment in accordance with the invention as 
shown in Fig.6, the type of application is checked first, and if the application belongs to the 
group, step 5 will be performed. If the application in the opened window is not one of the 
group, it is checked whether the file extension of the file displaymg information in the 
window is one of the group. If so, step 5 is performed. If not, it is checked whether the 
information displayed in the window is moving. If so, step 5 will be performed, if not, step 6 
will be performed. It is possible to alter the order of the steps 2 to 4, or to proceed to step 5 
only when two or all of the steps 2 to 4 are answered affirmatively. In the latter situation, 
there is a greater chance that the window will be highlighted only if the information in the 
window is non-synthetic. In step 5 it is checked whether user input is received, indicating 
whether the window opened should be enhanced or not. This user input overrides the 
automatic determination. If the user information indicates that the window should be 
enhanced, the wmdow will be enhanced in step 7. If the user information indicates that the 
window should not be enhanced, the window will not be enhanced and the process will 
proceed with step 1 . If no user information is received with respect to the window, the 
window will be enhanced in step 7. In step 6 it is detected whether user input is received, 
indicating whether the window opened should be enhanced or not. This user input overrides 
the automatic determination. If the user information indicates that the window should be 
enhanced, the window will be enhanced in step 7. If the user information indicates that the 
window should not be enhanced, the window will not be enhanced and the process will 
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proceed with step 1. If no user information is received with respect to the window, the 
window will not be enhanced and the process returns to step 1. 

One way of selecting a window for highlighting may be the following. The 
user sees a window (application) which contains content suitable for highlighting. The user 
5 presses a certain predefined "hotkey" on the keyboard (e.g. CRTL-H) and left-clicks with the 
mouse in the application to be highlighted. The hotkey is necessary to distinguish between 
regular left-clicks by Ihe mouse and left-clicks intended for the highlight application. 

When a user clicks the mouse, the OS (operating system) sends a "Message" 
(System Call) to the Application (that is the application with the window which has been 
1 0 clicked) to inform the application that the left mouse button has been clicked. 
5 Microsoft Windows has the option to install a so-called system hook, which 

Jil intercepts the messages sent from the OS to the application. The HiLite application can 

W install a system hook, which will normally pass all these System Calls to the application (e.g., 

Sj the system acts completely normal), unless the "hotkey" has been pressed. Note: "Hotkeys" 

L 15 are a feature of windows. The HiLite application can define a certain HotKey. Whenever this 
C key combination is pressed, the HiLite application will be informed. When the HotKey is 

[ I pressed, the "hook" will start analyzing all system calls. Whenever it finds a "left-mouse 

O click system call" it will determine the ID number of the window (in Windows termmology, 

the window handle, hWnd) and will pass the Id information to the HiLite application. This 
20 specific system call will not be passed to the application (because it was meant to be handled 
by the "hook"). The HiLite application will determine if the Window ID was fi-om the top 
level window. If not, the top-level wmdow will be determined (there are Windows API calls 
for this). Note that an application normally consists of several windows on top of each other. 
Some are visible, some are not. There is always a child-parent relation between the windows. 
25 This will be used to determine the visible top-level window for HiLiting. 

The HiLiting application can also detect if other applications are partially 
hiding the view of the selected window. This can be done by querymg all application 
windows (there is a Windows API call for this) and asking for position, size and z-order (=in 
front of another window, or behind another window). If another window is partial in front of 
30 the HiLited window, the HiLite application may decide not to HiLite, or to "break" the 
HiLited window into two different blocks (e.g. it is not a square window anymore). 

The same technique can be used to track movement of HiLited windows (e.g. 
when the user moves a certain application, the OS sends system messages to the application 
(e.g. "move to this position"). When the "hook" sees one of these messages, it will determine 
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the Window ID and send this information to the HiLite application. The HiLite application 
will determine if this window (or any of its children) are HiLited. If so, the Monitor will 
receive updated information of the area to be HiLited. 

IE and Netscape allow for loading plug-ins which have the option: get detailed 
knowledge of the browser system. The plug-ins can determine where pictures are positioned 
on a page (because pictures occur as jpg or gif files in the HTML code). These coordmates 
can be transferred to the HiLite application. The HiLite application should keep track of the 
movement of the browser application and the scrolling of the window. (The part to be 
HiLited is only a part of a window.) An additional problem is that not all pictures are 
rectangidar. Transparent backgrounds are used to make arbitrary shapes. These should not be 
HiLited. 

In Windows, one can get access to the frame buffer (which contains the 
picture which is displayed on the monitor) by using DirectX API (defined by Microsoft) or 
by installing a Filter driver between the Graphics Driver and OS/Graphics Display interface. 
The Filter driver is also able to allow access to the firame buffer. 

By scarming the frame buffer (completely, or areas suggested by the HiLite 
application, because it is discovered that a certain application just started, and it has its 
window at a certain position), the HiLite application can decide that this is an interesting area 
to HiLite. The HiLite application could use its information of all Windows to determine the 
exact area. 

A selection criterion could be a certain noisiness or edgi-ness of the content, 
contouring algorithms (pictures generally do not have rectangular contours, Windows does). 
In addition to static pictures, movement can be detected. 

Figs. 7 to 9 show an embodiment of a detection circuit for detecting movement 
in an area in accordance with the invention. 

In Fig. 7, the detection circuit DEI comprises a memory MEM in which the 
part 1, 2, 3 of the display information DI or a portion of the part of the display information DI 
is stored as first data DI at a first instant. A comparator COM 1 compares the first data DI 
with second data which is the display information DI of the part or the portion of the part of 
the display information at a second, later, instant, for example, one frame or 0.1 second later. 
The comparator COM 1 supplies difference values DIF to a processing circuit COM 4 which 
supplies the control signal CIl, which indicates whether a difference DIF between the stored 
display information DI and the corresponding display information at the second instant 
exceeds a limit value LV, This difference may be determined data word by data word, and if 
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enough differences occur, it is likely that the display information DI in the area shows a 
moving image (for example, a movie) and the control signal CIl is activated. The memory 
MEM may be the video memory of the graphics or video adapter GA. The comparator COM 
1 and the processing ckcuit COM 4 may be a suitably programmed processor, for example 
the microprocessor PRO already present m the computer PC. In an alternative embodiment, 
the detection circuit DEI may be located in the monitor MON. 

In Fig. 8, the detector DE2 comprises a memory MEM in which the part 1 , 2, 
3 of the display information Dl or a portion of the part of the display information DI is stored 
as first data Dl at a first instant. A comparator COMl compares the first data Dl with second 
data which is the display information DI of the part or the portion of the part of the display 
information at a second, later, instant. The comparator COMl suppUes difference values DIF 
to a circuit ABS for determining an absolute value ADIF of the difference values DIF. A 
summer SUM sums the absolute values ADIF of the differences of corresponding data words 
of the first and the second data to obtain a sum SDIF. A further comparator COM2 compares 
the sum SDIF with a limit value LV. If the sum SDIF is larger than the limit value, the 
differences of corresponding data words at different instants are large enough to have a 
reliable indication that the video information in the window is moving information. This 
embodiment of the detector DE2 may be implemented in either the computer PC or the 
monitor MON, and may comprise dedicated hardware or a suitably programmed processor. 

In Fig. 9, the detector DE3 comprises an integrator INT which receives the 
display information DI to supply an intensity value DIN of a line or a sum of Imes in the area 
1, 2, 3. The integrator INT integrates the display data DI during a time interval corresponding 
to a part of the area or the complete area to obtain the intensity value DIN. A sample-and- 
hold circuit SH stores the determined intensity value DIN at a first instant as the stored 
intensity value SDIN. Or in other words, the intensity value determined during the time 
interval starting at a first instant is stored. A comparator COM3 compares the stored intensity 
value SDIN with a fiirther intensity value DIN of a corresponding line or a sum of lines in the 
area at a later instant to supply the control signal CI3, which indicates whether a difference 
between the stored intensity value SDIN and the fiirther intensity value DIN exceeds a limit 
value LV. In this way, the comparator COM3 compares the integrated intensity of 
corresponding parts of the area at successive periods of time (for example, for successive 
frames), and determines if this difference is larger than the limit value LV. If the difference 
exceeds the limit value LV, it is very Ukely that the display information DI displayed in the 
area represents moving pictures. 
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It should be noted that the above-mentioned embodiments illustrate rather than 
limit the invention, and that those skilled in the art will be able to design many alternative 
embodiments without departing from the scope of the appended claims. 

In Ihe claims, any reference signs placed between parenthesis shall not be 
construed as limiting the claim. The verb "comprising" does not exclude the presence of 
elements or steps other than those stated in a claim. The invention can be implemented by 
means of hardware comprising several distinct elements, and by means of a suitably 
programmed computer. For example, the detection circuits DEI and DE2 in Figs. 2 and 3, 
respectively, and the enhancement circuit EM2 in Fig. 3 need not be dedicated hardware 
circuits, but may be implemented as a suitable software algorithm running on the processor 
PRO of the computer PC. In the device claim enumerating several means, several of these 
means can be embodied by one and the same item of hardware. 

The brightness enhancement can be controlled by the display load as a 
function of time. The display load is the relative number of cells (pixels) which is on in a 
picture. The enhancement factor of tihe extra amount of light in the window to be enhanced 
may be selected to be the reciprocal value of the display load. The display load may first be 
filtered to obtain an average value over time. The maximum amount of light in the window 
may be limited (for example, to 1000 nits). 

Instead of Microsoft (R) Windows, other operating systems having a graphical 
user interface operating with windows can be used, such as OS/2. 



